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Abstract 

The gauge theory of spin glasses and statistical-mechanical formulation of error- 
correcting codes are reviewed with an emphasis on their similarities. For the gauge 
theory, we explain the functional identities on dynamical autocorrelation functions 
and on the distribution functions of order parameters. These functional identities 
restrict the possibilities of slow dynamics and complex structure of the phase space. 
An inequality for error-correcting codes is shown to be interpreted to indicate non- 
monotonicity of spin orientation as a function of the temperature in spin glasses. 
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1 Introduction 



The mean-field theory of spin glasses is well established [1]. There exists a low- 
temperature spin glass phase with complex structure of the phase space with 
such characteristics as replica symmetry breaking, many valleys and slow dy- 
namics. In contrast, investigations of realistic finite-dimensional systems have 
been hampered by difficulties of analytic treatment, and numerical methods 
have been the predominant tool of research. An interesting exception is the 
gauge theory which exploits gauge symmetry of the system to derive many 
exact /rigorous results such as the exact expression of the internal energy, a 
bound on the specific heat and various identities for correlation functions [1,2]. 

It may be a little bit surprising that this gauge theory is closely related with 
some problems of information processing such as error-correcting codes and 
image restoration. Suppose that one transmits digital information (a bit se- 
quence) through a noisy channel. An important goal of error-correcting codes 
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is to devise an efficient way to encode and decode the information in order 
to remove as much noise as possible from the output of the channel. This 
problem can be rewritten as an Ising spin glass [3]: A bit sequence is naturally 
expressed as an Ising spin configuration, and noise corresponds to randomness 
in exchange interactions. Then the statistical inference of the original infor- 
mation, given the noisy output of the channel, is found to be equivalent to the 
statistical mechanics of an Ising spin glass. 

A similar analogy exists between image restoration, in which one tries to 
remove noise from a given digital image, and the statistical mechanics of a 
spin system in random fields. It is the purpose of this contribution to review 
the gauge theory of spin glasses and information processing problems, error- 
correcting codes in particular, with emphasis on the similarities between these 
superficially quite different fields [1]. 

The gauge theory of spin glasses and error- correcting codes are reviewed in 
sections 2 and 3, respectively, with a few new results and viewpoints. Summary 
is given in section 4. 



2 Gauge theory of spin glasses 

The gauge theory of spin glasses is a powerful theoretical framework to derive 
a number of exact / rigorous results on models of spin glasses using gauge sym- 
metry of the system. It does not answer directly the question of the existence 
or absence of the spin glass phase in finite dimensions. Nevertheless this the- 
ory leads to various results which set strong constraints on the possibilities of 
the existence region of the spin glass phase in the phase diagram and other 
non-trivial conclusions [1,2,4]. 



2.1 System and its symmetry 

Let us consider the ± J Ising model with the Hamiltonian 

H = -J2 J ij S i S j, (!) 

(ij) 

where J^-, a quenched random variable, is either J (with probability p) or — J 
(with probability 1 —p). The sum over (ij) runs over appropriate pairs of sites, 
such as (but not restricted to) nearest neighbours on a regular lattice. This 
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probability distribution can be written in a compact form as 



where Tjj(= ±1) is the sign of J^-. It is easy to check that (2) is equal to 
p if Tij = 1 and is 1 — p when r^- = —1 if we define the parameter K p by 

e -2K p = 

The Hamiltonian (1) is invariant under the gauge transformation 

S'j > iS'jCj, t/jj > Jij&iCTj, (3) 

where {<7j} is a set of Ising variables fixed arbitrarily at each site. 
2.2 Exact energy 



It is possible to derive the exact expression of the internal energy, averaged 
over the probability (2), under a simple condition between the temperature 
and the probability p. The definition of the internal energy is 



{Tij} (ij) ^{Si} 6 



[^Enw-^r, (4) 



where the square brackets [• ■ •] denote the configurational average. The gauge 
transformation (3) just redefines the running variables {r^} and {Si} and 
therefore does not affect the value of the internal energy. Thus (4) is rewritten 

as 

e K vYl, Ti i u ^i J^s He K ^ TijSiSj 
^ = V (2 cosh K p ) n b ' J2 s e K E^^ ' ( 5 ) 



where Nb is the number of bonds in the system and K = f3J. Since the value 
of (5) is independent of the choice of we may sum the right hand side 
of (5) over all possible values of {<7j} and divide the result by 2^ without 
changing the value of [E] : 

\E] = - VVe^S w . T.sHe K ^s lS3 
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Fig. 1. Phase diagram of the ±J model and the Nishimori line (dashed). 

It is clear here that the denominator on the right hand side cancels with 
the sum over a if K — K p , which makes it possible to evaluate the energy 
explicitly: 



[ E] = - W He K £ Tij s ' Sj 

[ 1 (2coshK p )^2^ V s 

= I (-—\ We x £^ 5 ^ 

(2 coshK p ) N B2 N \ dp) V s 



(2coshK p )^2 w 
= -iV B Jtanh K. 



-|) £(2 cosh A') 



N B 



(7) 



The only condition for the above manipulations to be valid is the relation 
K = K p to relate the temperature with the probability, defining a line in the 
phase diagram, the Nishimori line shown dashed in Fig. 1. Note that the lattice 
structure, range of interaction, or the spatial dimension are all arbitrary. So 
the present exact solution (7) is very generic. 

Similar arguments lead to an upper bound on the specific heat and various 
identities for static correlation functions under the condition K = K v . 



2.3 Absence of slow dynamics 



An interesting relation can be established for dynamical correlation functions 
using the gauge theory [5,6]: 



(Si(t w )Si(t + t w )) K 



{Si(Q)Si(t)) 



(8) 
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tw tw ~\~ t 

Fig. 2. Two measurement times of the dynamical correlation function on the right 
hand side of (8). 

The right hand side is the equilibrium autocorrelation function at temperature 
K = K p . The left hand side is a non-equilibrium correlation function starting 
from the completely ferromagnetic state at time 0. The first measurement 
is performed after waiting time t w and the second at t w + t (Fig. 2). If the 
systems shows anomalously slow dynamics, the non-equilibrium correlation 
function, the left hand side, will depend on the measurement starting time (the 
waiting time t w ) as it actually happens in the spin glass phase of the mean-field 
model. Otherwise, the system relaxes quickly to equilibrium, implying time- 
translation invariance, or equivalently, independence of the left hand side of 
t w . The identity (8) proves that the latter possibility is the case since the 
right hand side does not depend on t w . Therefore, any spin glass system does 
not show slow dynamics on the line K = K p . The proof of (8) requires some 
background of the kinetic Ising model and we refer the reader to the original 
papers. 

Note that the above argument is not a proof of the absence of a spin glass 
phase in general. What has been shown is that a phase with slow dynamics 
should exist, if it does at all, away from the Nishimori line. 



2.4 Distribution functions of order parameters 



An interesting functional identity P m (x) = P q (x) can be proved under the 
same condition K = K p [7]. Here, P m (x) is the distribution function of mag- 
netization and P q (x) is the distribution of spin glass order parameter, 



x 



(9) 



and 



(10) 



The proof of the identity P m (x) = P q {x) is relatively straightforward if we 
apply the gauge transformation to the expression (9) and use the condition 
K = K p . After the gauge transformation, (9) is expressed as 
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Pm{x) 



(2 cosh K p ) n b2 



N 



EaEst(x--k Ei S'itTi) e ^E w^E^s, 



(11) 



The last expression is equivalent to (10) if K — K p . 



Since the distribution function of magnetization P m (x) is always composed of 
at most two simple delta functions located at ±m, the identity P m (x) = P g (x) 
implies that the distribution of spin glass order parameter P q (x) also has the 
same simple structure. This excludes the possibility that a complex structure 
of the phase space, which should be reflected in a non-trivial functional form of 
P q (x), exists on the line K = K p . Thus a spin glass (or mixed ferromagnetic) 
phase of the mean-field type, if it exists, should lie away from the line in 
the phase diagram. This observation reinforces the conclusion of the previous 
subsection on dynamics that there is no mean-field type spin glass phase on 
the Nishimori line. 



3 Error-correcting codes 



The arguments in the previous section on the gauge theory of spin glasses 
are in close formal similarity to some problems of information processing. We 
demonstrate it with error- correcting codes as a typical example [1]. 



3. 1 Transmission of information 



The goal of the theory of error-correcting codes is to find efficient methods 
to infer the original information (bit sequence) from the output of a noisy 
transmission channel. For this purpose it is known to be necessary to introduce 
redundancy before sending the information through the channel, a process 
called encoding (or coding), see Fig. 3. At the receiving end of the channel, 
one decodes the signal to retrieve the original information making full use of 
the redundancy. Shannon's channel coding theorem sets a lower limit on the 
redundancy in order to retrieve the original information without errors, given 
the noise strength of the channel. 
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Fig. 3. The original information of N bits is encoded to a binary sequence of 
Nb(> N) bits which is then transmitted through the noisy channel. The output of 
the channel is decoded to infer the original information. 



3.2 Ising representation 



It is convenient to formulate the problem in terms of Ising spins as natural 
correspondences exist between a bit and an Ising spin (0 and 1 corresponds to 
Si = (—1)° = 1 and Si = (— l) 1 = —1, respectively) and between the mod- two 
addition and the multiplication (e.g., 1+1=0 is equivalent to (— l) 1 x (— l) 1 = 
( — 1)° = 1) [3]. Suppose that the original information one wishes to send is 
an Ising spin configuration Redundancy can be introduced by generating 
a set of many-body Mattis-type interactions [8] from the spin configuration, 
J?jk- = CiCjCfe • • •• Clearly the number of interactions N B is larger than the 
number of Ising spins N, implying redundancy. A specific choice of the set of 
sites to form interactions {ijk ■ ■ ■} corresponds to a specific code. A search 
for a good code is one of the main targets of coding theory, but we do not 
review this problem here and continue the argument without specifying the 
set {ijk ■ ■ ■}. 

The encoded information J° = {Jfj k ...} is transmitted through the noisy chan- 
nel. Let us focus our attention on a binary symmetric channel in which the 
input Jij k ... is flipped to the opposite sign Jijk- = — J^k- with error proba- 
bility p. The task is to infer (decode) the original information {^} from the 
noisy output of the channel J = {Jijk-. }- Note that frustration exists in the 
set of interactions J although J° (which was generated by the Mattis rule) 
is not frustrated. Finite-temperature statistical mechanics of the Ising spin 
glass with interactions J serves as a useful theoretical platform to decode the 
message as shown in the following [3,9-12]. 



7 



3.3 B ay esian formulation 



The first step is to express the noise characteristics using the conditional 
probability as 

P(J ijk ...\J ijk ...) = P(Jijk-\£i£j£k ■■■) = 2 cQsh n ( 12 ) 



which is equal to p if Jijk- = —J%k- an d 1 — P otherwise if we define (5 P as 

(13) 



D 2P P _ 1-P 



V 



Note that (3 P is essentially equivalent to K p appeared in the previous section, 
the only difference being that p and 1 — p are exchanged. Assuming a mem- 
oryless channel, in which each bit is affected by noise independently of other 
bits, the conditional probability of the whole spin configuration is 



where Nb is the size of the set J. 



Equation (14) represents the conditional probability of the output J given the 
input To infer the original information, we need the conditional probability 
of the input £ given the output J. The Bayes formula is useful for this purpose: 

P(J\<t)P(<t) 

PHJ> = E.WW (15) 



where a is a set of dynamical variables used to infer £. 

We need the explicit form of the distribution function of the original informa- 
tion -P(cr), called the prior, to use the Bayes formula (15) to infer a given J. 
It can reasonably assumed that -P(cr) is a constant, a uniform prior, because 
one often compresses information before encoding, which usually generates a 
very uniform distribution of Os and Is. Then the cr-dependence of the right 
hand side comes only from P(J\cr), and thus we find 

P(cr\J) oc P(J\cr) oc exp(/5 p Jijk-^j^k- • •)■ ( 16 ) 

{ijk-} 
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The right hand side is the Boltzmann factor of the Ising spin glass with many 
body interactions which are quenched since J is given and fixed. Therefore 
the problem has been reduced to the statistical mechanics of the Ising spin 
glass with the effective coupling f3 p . 



3.4 Statistical inference 



There are two typical methods to infer the original information (spin config- 
uration) from the conditional probability (16) called the posterior. One is the 
MAP (maximum a posteriori probability) method in which one chooses the 
configuration cr that maximizes P(<r\J). This is equivalent to the ground-state 
search of the Ising spin glass 

H — - Jijk-ViVjVk- ■ ■ (17) 

{ijk—} 



according to (16). Another is the MPM (maximizer of posterior marginals) 
method; one first marginalizes the posterior with respect to <7j 

P{a i \J)= £ P{a\J) (18) 



and chooses ex; to maximize this marginal probability. If we write the inferred 
value as we have = arg max CT .P(<7j| J). Such a process is carried out at 
each %. The MPM is equivalent to looking at the sign of the local magnetization 
& = sgn(<7j), where the thermal average is taken using the Boltzmann factor 
(16) of the Ising spin system with effective coupling f3 p . The MAP may be 
regarded as the same method £j = sgn(<jj), the only difference being that the 
thermal average is now evaluated at the ground state. Thus these two meth- 
ods may be called the ground-state decoding (MAP) and finite temperature 
decoding (MPM), respectively. 

An important measure of decoding performance is the overlap of the decoded 
information (spin configuration) and the original true information {&}, 



M — — 
N 



(19) 



where the square brackets denote the configurational average with respect to 
the distribution of noise (or equivalently, quenched randomness in interac- 
tions). Perfect decoding result gives M — 1 whereas a random result yields 
M = 0. A larger M represents a better result. 
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Fig. 4. Overlap function of the temperature 



It is useful to replace the parameter /3 P in (16) with a general (3 because, first, 
the MAP {(3 — > oo) and the MPM {(3 = (3 P ) can be treated within the same 
framework and, second, one sometimes does not know the correct error rate 
p (and its function j3 p ) in applying the MPM. The overlap is then a function 
of (3 and p. It turns out that the overlap M(f3,p) as a function of (3 is not 
monotonic but reaches the maximum at (3 p (Fig. 4) [10]: 



This non-monotonicity is not surprising intuitively because the overlap has 
been defined as a measure of average bitwise performance (19) and the MPM 
has been designed to maximize the bitwise posterior (18). The proof of the 
inequality (20) will be given in the next subsection. 

3.5 Interpretation as the Ising spin glass 

The inequality (20) has the following interesting interpretation in the context 
of spin glasses. Let us consider the problem in terms of the ferromagnetic 
gauge, £j = 1 (Vz). Generality is not lost by this restriction because the prior 
has been assumed to be uniform, P(cr) =const, and thus all the original con- 
figurations have the same statistical properties, leading to the same average 
decoding performance M(/3,p) for any choice of 

In the ferromagnetic gauge the overlap M(/3,p) represents the average of the 
difference between the numbers of up spins and down spins: 



It should also be noticed that (16) is equivalent to the Boltzmann factor of 
the ± J Ising model with many body interactions only for the ferromagnetic 



M{f3,p)<M(f3 p ,p). 



(20) 



M([3,p) = ±- =[8gnfo>]. 



(21) 
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gauge, strictly speaking. The reason is that the interaction Jyfc...(= £i£j£,k ■ • •) 
is equal to +1 with probability 1 — p and to —1 with probability p only in 
the ferromagnetic gauge. Thus the effective inverse temperature (3 P in (16) 
represents the condition that the system is on the Nishimori line (3 P = (3 in 
the ferromagnetic gauge (with p and 1 — p exchanged from the notation of 
section 2). 

To prove (20) in the ferromagnetic gauge, we first write (21) explicitly: 



I Ml 



E^J^— . (22) 



(2 cosh ft) |E„ff,e"E 



Then, by applying the gauge transformation (with gauge variable rji) and 
evaluating the upper bound by taking the absolute value of the summand, we 
find 



IWI 



2^(2 cosh /y^^ v ""*|fa>/ 



The last expression is easily seen to be equal to [sgn(<7j)fl ] = M(/3 p ,p) if we 
apply the gauge transformation to [sgn(<7j) J g J. 

The inequality (20) is interpreted that the number of up spins, relative to 
the number of down spins, is a non-monotonic function of the temperature. 
The number of up spins reaches its maximum on the Nishimori line and then 
decreases as one further lowers the temperature (Fig. 4). In this sense, the 
spin state on the Nishimori line is more ordered than the states at any other 
temperature with the same p. 

This is a highly non-trivial result. As one lowers the temperature from the 
paramagnetic phase, the number of up spins becomes to exceed that of down 
spins as soon as the system enters the ferromagnetic phase (point A in Fig. 5) if 
one imposes the boundary condition such that the overall inversion symmetry 
is broken for the system with two-body interactions. Q The number of up spins 
reaches the maximum on the Nishimori line, where, in some sense, thermal 
fluctuations are balanced with geometrical frustration (point B). Then, as one 
further lowers the temperature, the spin states start to capture the detailed 
structure of bond configurations, which were masked by thermal fluctuations 



1 Note that the argument in the present section applies to the usual case of two- 
body interactions as well. 
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Fig. 5. The spin state is more ordered at point B than at A or C. 

above the Nishimori line, and the number of up spins starts to decrease at 
lowering the temperatures (point C). 

It should be remembered here that the magnetization m(/3,p) = [(cj)^], not 
the overlap M((3,p) = [sgn(<Tj) / g], is of course expected to be a monotonic 
function of the temperature. Only the number of up spins M((3,p), which 
ignores the spin-size reduction due to thermal fluctuations (| (cjj) | < |sgn(cr i ) | = 
1), is non-monotonic. We may therefore conclude that the Nishimori line marks 
the crossover between the purely ferromagnetic region (where M increases as 
the temperature is decreased) and a randomness-dominated region (where M 
decreases). This observation is reinforced by the fact that the ferromagnetic 
order parameter m is equal to the spin glass order q on the Nishimori line 
as can be derived from the functional identity P m (x) = P q (x) mentioned in 
section 2: We reasonably expect that the ferromagnetic order dominates m > q 
above the Nishimori line and the opposite is true m < q below. 

Another interesting and useful aspect concerning the relation (or equivalence) 
between the Ising spin glass and error-correcting codes is the simplicity of 
phase space and the absence of slow dynamics on the Nishimori line discussed 
in section 2. Practical algorithms of decoding are generally implemented as 
iterative solutions of TAP-like equations for local magnetizations. Such itera- 
tions can be regarded as discrete-time dynamics of the Ising spin glass. And 
the iteration is often carried out on the Nishimori line to achieve the best per- 
formance in the sense of maximum overlap M. Then, if the system shows slow 
dynamics, the iteration may not converge to the desired result in a reasonable 
amount of time. The results in section 2 guarantee that this does not happen. 

□ 



There still exists a problem of appropriate choice of the initial condition in 
practice [13]. 
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4 Summary 

Exact /rigorous results have been presented on static and dynamical properties 
of the Ising spin glass using the gauge theory. These results have been shown 
to restrict the possibilities of complex structure of the phase space and slow 
dynamics of the mean-field type. We have presented a Baysian formulation of 
error-correcting codes that has a very close formal similarity to the Ising spin 
glass. The best bitwise inference of the original information has been shown to 
be achieved by a finite-temperature decoding corresponding to the Nishimori 
line derived in the gauge theory of spin glasses. We have seen that an inequality 
on the overlap, which is a measure of decoding performance of error- correcting 
codes, has an interesting interpretation under the context of spin glasses that 
the average spin orientation is not a monotonic function of the temperature. 
The spin ordering, in the sense of the number of up spins (ignoring the spin 
reduction due to thermal fluctuations), takes its maximum value not at T = 
but at a finite temperature corresponding to the Nishimori line. Thus the 
system of Ising spin glass is most ferromagnetically-ordered not in the ground 
state but at this finite temperature even within the ferromagnetic phase. 
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